OOCUHBHT &ESOHB 



BD 092 145 

AUTHOR 
TITLE 

IHSTITUTIOH 

PUB DATE 
MOTE 

EDBS PRICE 
DESCRIPTORS 



IR 000 705 

SchraiB, Wilbur 

Notes on Case Studies of Instructional Media 
Projects. 

Stanford Univ., Calif. Inst, for Coaaunication 

Research. 

Dec 71 

i*3p. 

MP-$0.75 HC-$1.85 PLUS POSTAGE 
*Case Studies; Evaluation Methods; Experinents; 
♦Instructional Media; Instructional Technology; 
Investigations; Media Selection; ♦Research 
Methodology; ♦State of the Art Reviews; Surveys 



ABSTRACT 

The case study is the aethod of choice in studying 
instructional aedia projects. In contrast to experiaents, which 
analyse the effect of a on b, and saaple surveys, which ask "What is 
out there?", the case study asks "What happened?" and is both 
descriptive (unlike the expeciaent) and free of tiae (unlike the 
saaple survey). The objective of a case study — to illuainate a 
decision or set of decisions— is ideal for instructional aedia 
projects. Researchers can enter a situation after the fact and 
benefit froB their perspective in tiae. Central to the case study is 
t he resea rche r who should bring to the siiuatioh 'Historicar^ 
political science skills for a wholistic view. In case studies 
docuaents, interviews, observations, and secondary analyses are the 
aain data sources; and researchers are urged to aake greater use of 
docuaents, interview the right people, and aake observations acre 
objective. Addressing an audience and focusing on key decisions will 
help make a case study useful. Exaaples of both good and bad case 
studies of various types are appended. (WH) 
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NOTES ON CASE STUDIES OF INSTRUCTIONAL MEDIA PROJECTS 



If we tnay begin with a fanciful analogy » we can say that a 
case study la to an experiment as Gibbon Is to Euclid or Newton. 

Admitted the analogy Is fanciful* Writers of case studies 
are more likely to resemble a thoughtful reporter than the historian 
of the Roman Empire. But> on the other hand» not many experimenters 
closely resemble either the author of the classic geometry or of the 
laws of motion. And within the limits of social research* the 
distinction is useful: an e^eriment abstracts life toward laws; 
a case study organizes the details of life in search 6f pattetns 
and insights. 

In our preoccupation with scientific method we have tended to 
treat the case study as a country cousin of the experiment » something 
not to be shown in public by an aspiring researcher. To take this 
position, however. Is to Ignore a rich source of knowledge, useful 
where more rigorously scientific methods have not been ov cannot be 
applied, and a rich source of hypotheses for future testing. Other 
fields of learning have demonstrated its capability. The case study 
was the principal tool of Freud. It has been used almost wholly by 
cultural anthropologists to study primitive cultures. It has been 
used effectively in communication research by scholars like Merton 
(Mass Persuasion) and Cantril (The Invasion from Mars ) • And it has 
revolutionized the study of decision-making in fields like business 
administration. 
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It Is, therefore, not a weak tool of social research, and 
deserves some attention. 

Case studies and experiments 

the case study differs in an Important way from the experiment 
in purpose » focus, and method* 

Experimental research exists to test hypotheses and contribute 
to theory • The case study, on the other hand, exists to systematize 
evidence so as to suggest hypotheses for testing and, pending that, 
to provide a basis of fact and insight for possible application to 
decision making* 

The experiment, in almost every case, works with a situation 
that has been contrived and controlled by the researcher* The 
exceptions to this are the rare appearance of ''natural*' experiments, 
such as the diffusion of naturally occurring information which can 
sometimes be studied with quasi**experlmental controls; and "field" 
experiments, which deal with an already existing situation over which 
less than laboratory controls can be maintained* Therefore, there is 
less than a sharp line between the controls an experimental researcher 
and a case study researcher can exert over the material with which he 
works, but in general, the distinction holds: in perfprmlng an experi- 
ment, one can decide upon the situation and so far as possible control 
the elements within it; in making a case study, one accepts the 
situation and tries to comprehend it* 

Consequently, experimental research in its pure form must so far^ 
as possible exclude every detail, not controllable or measurable, that 
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Is related to the cause-effect relationship being tested. Case study 
method, on the bthev hand, is Inclusive rather than exclusive. It can 
afford to consider a large number of details, so as to consider their 
possible relation to a decision or a pattern of events. Its concern 
Is not to exclude what It cannot conuaand, but rather to avoid omitting 
eome detail that tnlght turn out to be Important In explaining what 
happened In the situation being studied* Consequently, in the field of 
Instructional media, an experiment tends to be a situation abstracted 
from a project; a case study tends to deal with a major decision, Its 
genesis and. Its apparent effect, or, more often, with the reasons for, 
the execution of, and the apparent effects of an entire project « 

The prpccdurcs of an experiment are rigidly directed and 
limited by experimental method derived from natural science. The 
procedures of a case study may Include any systematic way of gathering 
and organizing data. A case Is non-experimental but not necessarily 
non-quantltatlve. It may report and analyze experimental and survey 
data and relate these to decisions and other characteristics of the 
project. And there Is little excuse, in the present stage of Instructional 
media research, for a case study not to report and analyze cost figures. 

Thus, In the Festlnger-Katz (1953) taxonomy of research methods 
field studies, sample surveys, experiments In field settings, and 
laboratory experiments — case studies fall In the first category, although 
they may report evidence from the second and third, and occasionally 
from the fourth, also. In the Selltlz dichotomy (1959) — • exploratory 
and descriptive studies, versus studies testing causal hypotheses — case 
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studies ate clearly exploratory and descriptive, although concerned 
with causation, and prlTBarlly analytical To distinguish case studies 
from other exploratory and descriptive research, it will be helpful 
to consider their relation to sample surveys* 

Case studies and surveys 

The prime object of a sample survey is to establish reliability 
for a certain kind of descriptive data — usually the kind that can be 
reported verbally by representative members of a population* Its 
methods are sampling and correlation. The principal objective of 
Its analysis is to establish relationships within a population at 
a certain point in time* 

Time is thus an Important point of distinction between a 
survey and the other two methods we have been talking about* If 
a sample survey draws any conclusions about changes over time ot 
the effect of some event, at time a, on another event at time 
a 4- X, it must do so Inferentlally « That is, it can describe the 
television viewing habits of children at two different ages, and 
infer that a given child is likely to go through corresponding changes 
over the same age period* It can describe the behavior of a sample 
of farmers at different stages of deciding whether to adopt a new 
agricultural practice, and infer that any representative farmer is 
likely to go through those same stages of adoption* It can infer 
that one stage always precedes another, and is therefore, at least 
In that sense, a cause of the other* 

An experiment, on the other hand, is concerned with a restricted 
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and controlled segment of time between 8» which Is being studied 
as a cause, and r, which Is being studied as an effect. It has tunnel 
vision, In contrast to the wlde**angle vision of the survey. On the 
other hand, It can apply tests of Inferential statistics to Its 
cause-effect relationships, whereas the survey cannot; It can only 
establlshi with correlational statistics, relationships at a given 
time. 

case study Is centrally concerned both with time and with 
description. It seeks to record why a given decision was taken, how 
It was worked out, and what happened as a result. The decision may 
be one to establish a project for example, carry out an educational 
reform. A case study of any siase wll! deal with a number of decisions 
taken In the course of carrying out the original decision, will describe 
the situations In which they were taken and the procedures Involved In 
carrying them out, and the effects of doing so. It Is therefore free 
to cover a wide time span and to describe a variety of situations and 
relationships. But (except as it reports survey data) It cannot 
describe with the controlled reliability of a survey, nor (except as 
it reports expc^rimental data ) establish causal relationships over 
time with the controlled rigor of an experiment. 

In everyday terms, a survey is concerned with *'what is out 
there?'' a case study with **what happened?** and an experiment with 
"what is the effect of a upon b?" 

The essence of a case study 

There are at least as many kinds of case studies as there are 
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kinds of surveys and experiments. Yet» the essence of a case study, 
the central tendency among all types of case study » is that it tries 
to illuminate a decision or a set of decisions: why they were taken» 
how they were implemented, and with what result. 

A case study is therefore an effort to contribute to 
policy and decision making, rather than to science. All these 
distinctions blur. An experimenter, for example, may read a 
case study to dative hypotheses for scientific testing. A sample 
survey is used by a political candidate as a help in deciding 
how to conduct his campaign. A cross-media experiment may be used 
to help decide whether to teach by television or face-to-*face. But 
a case study is deliberately and dentrally designed to illuminate 
a decision, a policy, and a practice, and it will be useful to the 
extent that a reader can relate his own decision probleths to those 
described in the case. 

A complete report of an instructional media project is thus 
almost necessarily a form of case study. Thus, for example, in 
reporting upon the oducatlonal reform In El Salvador It Is necessary 
to go beyond the surveys of attitude among pupils, teachers, and 
parents, and the experimentally designed measurements of learning. 
The report will include these, and a great deal of other quantitative 
data on costs, attendance, production , retention of pupils, and so 
forth. But these must all be put into a framework which is 
relational and analytical and includes a great deal of description 
not derived from sample surveys, and much recording of events over 
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time not derived from experiment. Survey and experimental data are 
thus U6ed to contribute to the Interpretation of the whole project. 

A case study, therefore, need not be brief, and need not be made 
— as many such studies are In the course of a single brief visit 
Intended to gather evidence vaccuum cleaner style. It may be short 
or long, superficial or in depth. Instructional media cases can be 
studied not only through retrospective data, but also over time. 
A modern researcher, equipped with quantitative research tools, may 
decide to watch the development of a decision or a project through 
a long cycle, in the manner of a case study, meanwhile gathering 
Information through a variety of means. Including perhaps survey and 
..escperiment *. 

Thus, a case study will vary in length and detail according 
to the level of decision making to which it is intended to contrlbutei 
and the level of understanding it is Intended to serve. An example of 
an Intelligent, but very limited, study is Lefranc^s (1967) case on 
ITV in Niger. That study was made in one visit in 1966, supplemented, 
of course, by study of such documents as were available* It has been 
criticized for failing to note the flaw in the organization of the 
Niger project that later seemed to be responsible for its failure to 
develop into a national system, as had been originally intended. An 
example of a longer-term case study, in some depth we hope, will be 
the report of the French group of researchers who worked with the project 
for five years in Niger, and are now writing up their notes and data* 

To say it another way, the essence of the case study is history 
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and policy science. The scientist contributes where he has the datai 
but the historian and the policy scientist are In charge* 

Strengths and weaknesses of the case study 

As a tool of decision making, the case study las certain 
strengths and certain weaknesses* 

For one thing, It can enter a situation after the fact , 
and thus not lose valuable evidence. This Is what Merton did when 
he decided to study the remarkable success Kate Smith had In selling 
war bonds with a radio talkathon, and what Cantrll did when he decided 
to study an event that was not foreseen the panic resulting from 
a broadcast of a Hallowe'en play describing a fictional Invasion 
from Marsr On the other hand, the ability to enter late Is not always 
an advantage. All too frequently, a researcher enters the situation 
too late, when the early history has mellowed, some of the early 
problems and failures have been forgotten, and the skein of policy 
making can be restored only through the memories of men who have been 
too closely Involved to be objective about It* One of the most common 
fallings of case studies, when they fall. Is too much reliance on 
projecv administrators* When one enters too late Into the study of a 
project, or can devote too little time to It, then the administrators 
become the chief available custodians of Information and one Is unable 
to check up on their official recollections of what happened. 

Nevertheless, the ability to cover a sweep of time Is a real 
advantage In case studies. Consider, for example, the way that 
India's experience with the radio rural forum has been studied. The 
forum was first tried In 144 villages of Maharashta province, In a 
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^ ten-week pilot project In 1956. This was studied with some care> 
using an experimental design, and the report (Mathur and Neutath, 1959; 
Meurath, 1960) recorded a rather spectacular success. The participants 
were interested, they learned a great deal about the developmental 
projects that were the topics of the forums, and almost every experi** 
mental village reported that it had adopted several of the reconnsended 
innovations. On the basis of this trial, therefore, other states 
and countries could well afford to consider adopting the radio forums 
for their own rural development plans. And India decided, beginning 
in 1959, to extend the forums through the country, setting a goal of 
25,000 for the next five-year plan. 

If a case had been written at that time, it would have missed 
the main point of interest to development policy makers elsewhere « 
For when the forum program was expanded, certain very difficult 
problems appeared that had not been revealed by the pilot project; and 
the results were sufficiently discouraging to focus attention on the 
general problem of expanding pilot projects. A case study was tarltten 
in 1966, and, with that time perspective, was able to illuminate some 
of these expansion problems. We shall say more later about that 
particular case study. 

A sweep of time contributes greatly to perspective in 
instructional media studies. For example, it Is a great deal more 
interesting to examine the record of the Japan radio-televlslon-corres- 
pondence high school after six years than after its first year. Did 
the early popularity hold up? How did the student body change? What 



record did graduates make? Did the co^ts rise or fall? What new 
problems arose as the school grew from an exciting innovation to an 
established Institution? Similarly* while the report of the first 
year of the British Open University will be of great interest, a 
study in four or five years will be much more useful to other 
countries that might introduce the open university. The ability thus 
to record and interpret the history of a project is one of the real 
strengths of a case study. 

The vide angle lens of the case study extends not only over 
time, but over space. It can deal with the project as a whole » 
rather than a part of the activity abstracted from the whole. What 
it loses in detail* it gains in breadth. ThuSi it can deal with more 
than one or two interactive elements in a decision. It can reproduce 
the conditions with a fullness that will help a reader to decide 
whether he can relate that decision and its result to his own problems. 
In ather words* a well^-reaearched* well-written case has an air of 
reality that commends it to a practical policy maker. 

The trade off* in a sense* is between precision and completeness* 
controlled abstraction and less-well controlled life. Obviously* a 
case study cannot give the assurance one gets from an experiment that 
its results are reproducible. One can never be sure that the decline 
of the Roman Empire might have taken another course* given one of the 
alternative decisions Gibbon describes. But on the other hand* it 
is no longer possible — in fact* never would have been possible — - to 
experiment on the decline of the Roman Empire. It would have been 
possible to design an experiment on the expansion of a pilot project 



like the radio forums » and It would have helped to have mote than one 
such experiment » representing some sampling of vhat happened under 
different conditions. But these experiments vere not done; let us 
hope that some day the elements of expanding a project that lend 
themselves to experimental treatment vill be studied in that vay. 
But in the meantime^ and with a great deal less effort and cost| 
the case study method has stepped into the breach^ taken a broad and 
realistic view of the situation, and presented an interpretation which 
Is useful to other policy makers. 

What aspects of an Instructional media project lend themselves 
especially to case study, rather than experiment or survey? Kot 
precise conclusions on causality, certainly. Not precise descriptions 
of a population. Rather, history and summary analysis of vhat happened. 
Bow policy questions arose i how they were handled and why. The organi- 
sation and operation of the project. The financial history of the 
project, and the analysis of unit costs, operational and capital 
Investments. These are not always easy to obtain, but they are the 
kinds of information me has a right to expect from a case. 

Perhaps the greatest weakness of the case study is that it 
places an enormous responsibility on the researcher, rather than on a 
method. This is not to imply that an experiment is a mechanical task, 
that can be done as, for example, one changes a light bulb or the oil 
in his car. Far from that! It requires rklll and hard logical 
thinking. But the experimental method offers a certain specificity 
and a certain organization of data that is of great support to the 
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researcher. The scholar who makes a case study has far fever guides. 
He has less theory to go on, leas guidance as to what data he should 
exafflinst no statistical analysis of variance to help him Interpret 
what he finds ♦ Inatead of being encouraged to restrict his intake of 
information to certain comparable units » he is encouraged to range as 
widely as possible^ collect widely divergent information if it exists. 
Whereas the experlibenter Is in position to manipulate his subjects» 
the case study man must necessarily deal with a number of informants 
who he suspects are trying to manipulate him . And when the process 
of gathering data is completed » the experimenter has his hypotheses, 
and the theory behind them, and some significance tests, and a long 
tradition of what they mean, whereas the scholar making the case 
study has only his own insights to depend on. The case will stand 
or fall on how well he organizes and interprets what he has learned, 
and how effectively he presents it to readers who can make use of 
itf When information is scant or contradictory, and when the 
reputations of real people and the policies of real systems are 
at stake, this may be a rather frightening responsibility. 

Sources of data for a case study 

The data for a case study comes mostly from documents, inter** 
views, observation, and secondary analysis. 

The stereotype of a case study as a **site visit'* by experts is 
so pervasive that it may be well to point out that many case studies have 
benefited greatly by secondary analysis of quantitative data that Is avail- 
able but has not been analyzed in such a way as to contribute to the objective 
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of the study. This Is true especially of cost data» which can usually 
be obtained In raw fona but requites a great deal of processlngi 
Enrollment figures, dropouts, progression, retention, are usually 
available but need to be analyzed and set In historical terms or 
against total costs. In studying one of the "open university" types 
of project In which correspondence study Is combined with broadcasting, 
there Is often need for a rather subtle form of analysis to determine 
a full-*time equivalent figure so that comparisons with classroom teaching , 
can be made. Even in a project where no "research" on effectiveness 
has been conducted, there Is often data lying around which can be 
resurrected and analyzed in such « way as to get some idea of student 
performance. For example, in Samoa one case team was able to find 
old results of standardized tests that revealed what level bhe Samoa 
system had reached when the educational reform started. The raw data 
had been available f or a :number of years, but was never cited and never 
analyzed. 

Because a case study team typically spends so little time with 
a project, it is usually impossible to do such things as establishing 
reliability among observations and among observers things a researcher 
would feel it necessary to do in order to obtain reliable quantitative 
data from observations. However, if observation promises to be a major 
source of information about a project, there is no reason why a visiting 
team cannot deliberately build it into their plans. The stay can be made 
a little longer, or the team can be made larger. The schools or other 
observation points can be sampled randomly within some appropriate 
atratification. And observation forms are becoming available on which 
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It le podsible to establish observer reliability in a relatively 

short period of working together. For example » the Salvador form 

of analyzing classroom Interaction has proved readily usable In Mexico « 

Of course I a case study team usually spends the major part 
of its time en situ talking with people who have most experience with 
and most knowledge of the project, typically , a study team tries 
first — - preferably before arriving at the location to be studied 
to agree on its interpretation of the pre«-study information it has 
received and on the questions that seem most worth looking into» 
Then It usually meets as a team with its local counterparts » if any»* 
ceremonially with the highest officials with whom it deals, and with 
those officials who have the broadest views of the project and who 
should not be made io repeat their information for individual members 
of the team. Then the team members usually fan out to areas of their 
own specialty. The economist, if any, spends much of his time with 
the financial officers. The broadcaster, if there is one on the team, 
spends his time with the persons who make the programs. The educators 
talk with the curriculum people and the local school administrators. 
And so on. The team usually begins to operate in a kind of accordion 
pattern — looking into individual specialtie^> during the day, then coming 
back together to report results and check with each other on additional 
questions to be investigated. 

Heedless to say, this is a delicate kind of operation. The 
study team is trying to obtain a wholistic view of the project from 
people who, for the most part, know one area of it very well but may 



be much less well Informed about other areas* The team la trying 

to obtain a balanced and objective view of the project from peraons 

who I because of their own involvement and their own career a i are not 

likely to take a balanced and objective vlew> and In aome cases may 

not want to encourage aclose look at some part of the project* If 

team were In residence for a year > It would be In better position to 

weigh the evidence. But a visit of a few weeks places a difficult 

responsibility on the study team to translate a number of Individual 

viewpoints and sometimes contradictory pieces of evidence Into a 

picture of the whole, with fairness to everyone concerned. 

Coombs (1971) In a document prepared for teams undertaking case 

studies of educational development gave some wise, if Idealistic^ 

advice on this problem: 

'*It must be expected," (he wrote] "that knowledgeable 
Informants who have been deaply Involved In the program 
may often, In all honesty, have a rather narrow and 
specialized view of It and an unduly opt Imlstl: evaluation 
of Its performance. Obviously, their testimony Is 
crucially Important, for they are likely to have a 
uniquely Intimate knowledge of how the program evolved, 
the problems encountered, changes In the Initial plan, 
what has happened to earlier participants, etc. Yet to ' 
accept their story alone and at face value could result 
in a narrow and even distorted picture. In the extreme 
It could unintentionally make the case study a misleading 
propaganda piece, not a balanced and objective analysis* 
Therefore, it is incumbent on those preparing case studies 
to probe politely but persistently to get * the other side 
of the story • and to develop additional evidence that will 
put the matter in a broader perspective* 

"For this purpose it will be important to interview a 
mix of people who see the program from different vantage 
points, including some 'hard-nosed^ observers who know 
the program, see it in a broader context and have no 
direct involvement in its success or failure (such as 
well**lnformed analysts in the planning ministry or the 



finance mlnldtry). In the end the authord of the case 
etudy will have to weigh theee vatloue judgments and 
the assorted evidence and reach their own considered 
conclusions* 

It is Important in reaching such conclusions and in 
writing the report itself to distinguish between ^hard^ 
evidence, /soft' evidence, and ^informed opinion* » and 
between clear, firm, and well-documented conclusions and 
hypotheses or tentative conclusions (allowing room for 
possible alternative interpretations)/' 

\Jhat kinds of people must one be sure to interview? 
Obviously the people who have the best information* These will 
include the people responsible for the project and for different 
parts of it; the people who make the materials — broadcasts, films, 
teachers' guides, workbooks, etc.; the people who use these materials 

teachers and students; and the people best able to observe 
researchers, inspectors, educators who have visited, critics, 
parents when possible, local government officials, and others. 

To this should perhaps be added our own experience that it 
has always paid to interview the bitterest critics of a project, 
preferably before seeing the project itself. This may on occasion 
present a delicate problem of relationships with project sponsors 
and directors » but we have found that without hearing the chief criticisms 
in their most vigorous form, we have always felt uneasy as to whether 
we Iiad looked hard enough at the evidence we could gather. 

Perhaps the most undervalued source of data for a case study 
is documents . This is probably because of the unfavorable stereotype 
of a case study done by one man or a few men spending a short time 
"looking at" a project, coming to a quick decision, and reporting It. 
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But any conscientious case study team tries to assemble and read 
beforehand the chief relevant Information In prlnti and tries during 
the visit itself to collect all possible relevant documents* along 
with suggestions of other useful sources, to read later. This Is 
particularly Important when the project is in an unfamiliar culture. 
We can remember, for example, how useful it was to read Margaret 
Mead ^8 Coming of Age in Samoa , KeeslngU volume on Elite Communication 
in Samoa , and an all but burled dissertation on the history of the 
Samoan educational system, among other things, before going to Samoa 
to look at that educational reform project. 

Here are some examples of the kinds of documents that are likely 
to be useful: 

Background ~ histories (for example, no American who is 
not a specialist in Africa should be sent on an African case without 
reading at least a brief history of the peoples and nations of that 
continent); interpretations of the culture (likely to be as useful 
for a non*-8peclallst going from the United States to a case study 
in Latin America as to an apparently more different culture like 
Polynesian or Chinese or South Indian); geography (a good map of the 
country in which the case is located always repays study); demography 
(census figures, age breakdowns, population distributions); economics 
(Income and its distribution, occupations, national resources and 
budget); and descriptions of the educational system and the public 
communication syptem. 

Official documents — the laws covering education; parliamentary 
debates in connection with the reform; executive orders relating to the 



reform; relevant official stateffiente and public addtesBea* 

PlanninR documenta the national development plan and 
the budget Impleaentlng It} national educational plana » If any; 
reports of appropriate study or planning commissions (such asg 
for example » the Indian Educational Commission) . «' 

Pocuments on the educational system pre-*project planning 
(for example » if there was a World Bank study before the projects 
It will probably Include a great deal of useful economic data); 
organization charts and up^^to^-date descriptions of the system; 
data on private education in the area; enrollments over a period 
of years » including figures on dropouts and repeaters; descriptions 
of the curriculum and Its objectives; enumeration of teachers > 
amount of education and training they have had| and the system 
of teacher training* 

Documents on the project itself history, if available; 
organization chart, and changes in organization during the history 
of the project; statements of the goals and intended methods of 
the project (for example, the French volumes on the Ivory Coast 
project and the Bronson outline of the Samoa educational reform); 
organization char|:s (and previous organization charts) of the 
project; schedules both of producing and of delivering broadcasts 
or other materials; lists of equipment; de^^crlptlons, if any, 
of the workings of the project; copies of teachers? guides and of 
student work books; lists of textbooks, if any; descriptions 
and materials of in-service teacher training • 
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Audiovisual documents copies of broadcasts or films, 
If available, for more detailed study away from the project location* 

Cost data in print — education budget and project budget; 
list of capital expenditures, if available; previous cost analyses, 
if any. 

Research data in print — previous research reports, if any; 
summaries of test results if they exist; copies of feedback forms; 
testing schedules. 

Popular treatments significant magazine and newspaper 
reports of the project; articles criticizing or evaluating the 
project* 

Few projects are likely to be so rich in documents as this list 
implies, but a considerable documentary resource will be available in 
connection with almost any case study if the team will take the time 
to search it out and use it* 

A check list of content 

The nature of a case» its setting, and its magnitude will 
affect the information to be gathered. However, a representative 
check list of content may be useful, and for the purpose we might take 
a hypothetical case dealing with an educational reform, involving 
extensive use of instructional media. For that sort of case, there would 
ordinarily be an effort to cover the following kinds of informations 

1« The background 

the country, size, population, form of government, 
social structure, economic structure^ languages, educational 
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eystem. 
2« The project history 

--the problems the project was supposed to solve; the 
key decisions! when they were made, why i and under what 
circumstances; how the decisions were implemented; 
what difficulties arose during the development of the 
project » and what changes were made in the goals or 
patterns of the project; how the project organization 
changed, and yhy; what was the schedule for assembling 
resources I training participants, and launching the 
project. 

3. Organization of the project 

— where it belongs in the larger educational structure; 
its Internal organization, assignments, and lines of 
authority; what changes were made In the organization 
during the time of the project, and why. 

4. Project inputs 

— personnel required, training distribution, adequacy 
measured against needs; 

— amount and kind of technical assistance, its use and 
adequacy; 

— financial inputs: capital » operating, curve of expendi 
ture over time, sources of funds; 

— physical and technical requirements: buildings, 
equipment, transportation, etc.; 
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~ relation of Inputs to country^s other allocatlone 
" of reeourcee: to GNP and education budget^ to supply 

of trained teachera and technicians, to development in 
fluch industries as electronics » etc. ; 

— in retrospect, conclusions about adequacy of inputs, 
wisdom of devoting them to this project rather than 
others, likelihood of being able to continue project by 
means oi own resources, etc* 

5. Project throughouts: 

" the curve of project development over time; were 

the plans for this rate of development changed, and If so, 

why; 

--materials of the project: broadcast, print, other; 
their nature, number, content style; how they were 
produced, used, related to goals of project; 

— patterns of teaching: style, content, quality, strategies, 
relation to materials and to goals of project; 

— delivery systems: schedule of broadcasts, coverage, 
how non-broad'^ast materials provided, etc.; 

— support systems: utilization advice, in-service training, 
maintenance; 

— feedback systems: teachers* comments, student opinions, 
testing, observation* 

6« Project outputs 

— what actually got through to the students (e.g«, how many 
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man-hours of what kinds o£ teachlngi etc*); 
— • evidence of learning effects; 

evidence of effect on attitudes; 

evidence of effect on teaching; 
~ evidence of effect on attendance and dropouts; 

— evidence of effect on student participation and 
activity; 

evidence of effect on student aspirations » careers » and 
career plans; 

evidence of effect on goals » objectives, expectations of 
school system; 

evidence of effect on the wider community; 
~ evidence of contribution to development plans. 
7* Cost-effect analysis 

distribution of costs; unit costs; what was bought 
for how much; cost of project as compared to alternative 
ways of solving problem* 

8. Gross result 

what has happened, what is likely to happen to the 
projected reform; is It likely to continue when direct 
foreign assistance terminates, 

9. Analysis . 

— why results were as they were; 
reconsideration of key decisions taken vs. other 

possibilities; 

how the accomplishment measures against the goal; 
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— the Investment In retrospect? was It worth i&aklng/dnd 
could It have been better distributed or directed{ 

how the results of key decisions and stretegies in this 
project compare with those in other projects of similar 
type> and what conclusions can be drawn from this comparison; 
^- summing up local wlsdomt what would the people who have 
been closest to the project now do differently if they had 
another chance; 

— - integration of the conclusions: what can be learned 
from this experience « 

Sue&estlons for making a case study useful 

At the cost of occasionally repeating^ we are going to make 
a few suggestions to case study teams « 

1« Write it for the audience. 

A case study> as we have Indicated^ will find Its chief 
practical use among policy planners or administrators (or future 
planners or administrators) who can use it as a guide to policy and 
practice. Therefore^ It goes without saying that such a study should 
be readable by persons who are not fluent in social science or economic 
shorthand. 

2. Focus on key decisions. 

The planners and administrators who read the case will be 
faced by decisions more or less similar to those in the project 
described. They want to know why the key decisions were taken » under 
what circumstances, with what results. Pot example » when Samoa decided 
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to introduce ITV In all grades at once > rather than one grade at a tlmOi 
It made a fundamental decision required of all countries that Introduce 
television in a major way V What led the Samoa planners and administrators 
to take the decision they did? Were the conditions and pressures special 
in any way? What happened as a result? In retrospect^ was it a good 
decision? 

3. Relate the decisions in a particular case> where possible > 
to corresponding decisions in related cases » 

For example^ of the countries that used television as a 
prime mover in educational reform* Samoa was the only one that decided 
to introduce it in all grades at once. Why did Niger and El Salvador 
and the Ivory Coast decide to introduce it one grade at a time? Were 
their conditions essentially different from those in Samoa? How do 
the results compare? What we are saying is that it is not necessai^yi 
not even de6irable> to treat each case as an isolated study. Viewing 
It against the results of other cases will add important dimensions 
of richness and usefulness. 

4* Try to describe as fully and clearly as possible the' 
elements in a decision situation. 

This precept is too often honored in absentia « We have spoken 
of the iSamoa decision* Consider also some of the decisions in Niger: 
to send a French production team in to be chiefly responsible for 
making the programs; to use relatively untrained monitors rather than 
trained teachers in the classroom; to try to produce programs that 
would encourage a high level of activity among the pupils* The 
reader wants to know who took those decisions > and why. For example » 
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what were the alternatives to sending in a French teami and why was 
It decided to do that rather than> let us say i training Hlger 
people to do the job? And what can be said pro and con of that decision 
in view of what happened in Niger? What lay behind the decision to 
iiK>nltors7 If| as one can easily imaginei it grew out of a shortage 
of trained teachers or a desire to save money on the cost of training 
teachers* what considerations kept the project directors from using 
a comparative design monitors In some classrooms* teachers in others? 
More important K what did the monitors do? How did they teach? What 
was their responsibility, compared to what would have been expected of 
teachers? And how did it work out? In the case of the programs^ a 
reader wants to know a number of details: exactly what were the 
programs like? how were they made so as to encourage activity? what 
kind of activity? what went on In those classrooms? and what was 
the effect on learning? 

The reader of such a case study needs enough relevant details 
of these kinds to let him relate the decisions to his own situation. 

St Maximize the reliance on hard data and precise measurement. 

We have already said that there are usually more hard data 
available in any major project than most case study teams find* and 
that very often a secondary analysis will add a great deal to the 
report; furthermore * that it is not at all impossible to substitute 
more precise and quantifiable observation for the usual Informal 
observation by an individual expert* The addition of coat figures 
to a case study is something that no longer needs arguing for. We 
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have come out of a period when cost analysis was Introduced into very 
few case studies^ through a period when Its Importance was recognised 
by amateurs In economics who did amateur analyses » Into a period when 
the usefulness of a professional economist or cost analyst on a case 
study team Is widely recognized* 

6. Make clear which conclusions and Interpretations rest on 
hard data and confirmed Judgment^ and which are /^softer". 

Some years ago» it was common to write case reports either in 
the form of supposedly objective and factual reporting) rigorously 
purged of conclusions and interpretation » or in the form of 
professional Judgments » scarce in specific evidence but strong in 
the mystique of pi^ofessional expertise. In the field of Instructional 
media cases » at leasti it Is now safe to say that both those approaches 
have gone out of date. One wants more than reportlngi and one wants 
evidence for Interpreting facts and events » For that reason^ It becomes 
all the more important to be candid and explicit about the level of trust 
a reader can feel in what is reported and the conclusions drawn therefrom. 

?• Maximize the use of local experience. 

Reliance during a short visit on the persons who know the project 
best is a two-edged sword. It may open the way for those persons to make a 
good case for what they have done. On the other hand, an investigator can 
multiply his own efforts many-^fold by making use of the funded experience 
available on the scene. His problem in using this experience is what 
every investigator faces in gathering evidence > and there are familiar 
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ways to check one expetlence> one conclusion, against others. This 
kind of checking will require the case study team to consult nore 
widely than in administrative or official circles , and to seek out 
witnesses who are experienced and well**lnfortned| but have no need 
to defend the project or, for that matter, to attack lt» 
8« Stay a little longer. 

Probably no case study team has ever left a project completely 
satisfied that it has learned all it wants to know* Probably a team 
never will leave a case study site completely sated with information. 
But the longer it stays, the more opportunity it has to talk to the 
persons who were not in residence in the first week or two, observe 
what could not be observed in a hurried visit, and collect data which 
either did not come to attention or could not be gathered in a short 

time* 7;;^- 

The curve of a visiting team* a efficiency falls off sharply 
after two or three weeks, and the pull of other obligations grows 
notably stronger* But there is an answer to that, too* Some teams 
have found It very helpful to return a second time, after the data 
from the first visit have been analyzed and evaluated, and the 
gaps In the Information or the conclusions have become evident* Sometimes 
the second visit is scheduled for a time when the results of certain 
decisions can be more clearly seen, for example at the end of a school 
year when academic results become available* Given the current, well-- 
founded dissatisfaction with hurried, impatient case studies, it is 
entirely possible that the pattern of making a case study with two or 
more visits, separated by months or even a year, may become common* 
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9. Consider alternative cau6al patterns. 

When a field resear'cher performs a quasl-experlmenti he has 
a special obligation to consider alternative explanations for the 
effect he discovers* When he makes a case study » where causality is 
even harder to prove than in a quasi^-experimenti he has still more 
reason to consider all the elements in the causes-effect relationship* 
He reports a decision and the supposed result of it. But what other 
elements and forces might have helped bring about that result? Samoa 
Introduces television in all grades at once* Some results seem to 
be evident* But can we be sure that Introducing ITV in this way 
was the only, or even the chief, reason for the troubles that followed 
that decision? Could it have been partly the nature of the school 
system, or the nature of the children, or the level of training of 
the teachers, or the kind of television that was Introduced, or the 
changes in curriculum, or the fact that for a while two grades used 
the same television? These are very important questions for any 
other planner or policy maker* And consequently, before a case study 
team reports that X, Y, and Z happened as a result of decision A, it has 
a moral and professional obligation to try to eliminate, if possible, 
some of the alternate causes ( and to indicate the degree of confidence 
It feels in the causal pattern it reports, 

10# Do case studies, where possible, in related groups. 

The problem of matching cause to effect makes It all the more 
attiractive to plan case studies in groups of comparable projects. 
For example, Hirschman spent a year studying a number of World Bank 
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projects, not with the idea of reporting a tnass of details on each 
of them» but rather looking for common problems and common principles* 
He reported his findings In Development Projects Observed (1967) where 
he was able to deduce a number of principles about the special kind 
of Investment we call a development project » and the characteristics 
of a relatively successful project. Incidentally » one of the commonalities 
he observed about the set of projects that both he and the Bank regarded 
as among the best was that *^not one of the projects* « .was free from 
serious problems.** The way he went about this study left both him and 
his readers considerably more confident that the conclusions he drew 
and the propositions he advanced were not accidental or limited in 
their application. 

Ve hope to do something similar by studying In one group the 
chief uses of ITV for educational reform; in another groupi the 
chief useti of broadcast plus correspondence (the *'open university** 
or **everyman*sa:hool**) to extend education beyond the classroom or 
the campus; and in still another groups some of the chief Instructional 
uses of radio as a lower^^cost alternative to television. By examining 
projects thus together, ve hope to be able to speak more confidently 
about characteristics of the particular educational strategy, rather 
than the characteristics of a particular use of It in a particular place • 

Some types of instructional media case studies . 

The range of case studies in this field Is from a brief study by 
one man of a project where information is very scarce, to a team study 
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of a project which has been constantly researched and where a great 
amount and variety of Information Is already available on paper • In 
the former case, the product Is something like the article that might 
be produced by a competent reporter who stops by the project for a few 
days} In the latter, the case becomes, In effect, an overview of the 
entire project, a thoughtful interpretation of what was done and what 
was learned. Perhaps some examples may help* 

A study where information is readily available . In the 
autumn of 1971 a case report was written on the educational reform 
project in El Salvador. This project has been under study by a 
research team since it began, in fact since before the television 
went on the air or the new curriculum was introduced. In addition 
to measures of learning, attitude, and classroom interaction, there 
had been a complete cost stu'^^y and a continuing study of administrattve 
history and practices. The case study, therefore, represented an 
overview of the project and the findings at this particular point in 
the project history. This study illustrates very well that the case 
study form is still useful and appropriate when it is based on something 
©ore substantial than a short period of interviews and observation. 

The 70-page case study begins with some background on El 
Salvador: area, population, language, government, literacy ♦ occupations, 
the educational system and the steep pyramid of enrollment (only one in 
six children who enter the first grade goes as far as the seventh) . Then 
It turns to the origin and background of the educational reform. In 
El Salvador the use of instructional television for educational reform 



was already being discussed seven years before It began, the first 
feasibility study was made five years before the reform^ and two other 

i 

such studies, all In general agreement , were done before the project 
was financed. An active ETV Cotnmlsslon In El Salvador, the World 
Bank, AID, UNESCO, UNICEF and several other governments all made 
contributions to the planning and preparation, but finally It was 
serendipity, more than anything else the fact that the President 
of El Salvador was at Punta del Eete when President Lyndon B. 
Johnson proposed the Idea of an ITV pilot project In Latin America 
that made the reform possible. 

The study then takes up the extensive activity that had to 
take place between acceptance and realization of the reforin. The 
Ministry of Education was reorganized* The curriculum was revised 
and modernized. Every teacher who was to work In a classroom with 
television was given a year of retraining, both In the aubstance 
of the new curriculum and In new methods of teaching. Mew study 
i&aterldls were prepared for the classrooms. Supervisors wete trained 
to be advisers rather than Inspectors. A research evaluation team was 
Installed, Schools were modernized, and a building program begun. 
Television producers and technicians were trained, a studio equipped, 
and some programs recorded on tape. 

Then the case study describes the organization adopted for 
ITV in El Salvador, the television schedules, how the programs were 
made and used, and the problems with inadequate studio space. 

are on;|he . cpst total .^" 
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costs, year by year costs, sources of funds, ITV coats. Production 
in Salvador costs about $1,300 per hour» or $720 when averaged over three 
years, taking into account the necessary remakes* Television presently 
adds about 15 per cent to cost per student* Costs are iprojected Into 
the future, and estimated for possible extension to primary school 
and to out-of-school education* 

On the basis of the psychological and social research evidence, 
tentative answers are provided to such questions as; Do pupils learn 
any more from the new program? How much does television contribute to 
this learning? Has the reform affected the quality of teaching? What 
has been the effect on dropouts and on promotion? Does the new system 
equalize opportunities, or the reverse? What is the effect on student 
career and educational aspirations? What are the attitudes toward 
reform and toward ITV? 

The case study turns then to comparing the policy decisions taken 
in El Salvador with those in other countries that have introduced 
educational reforms built around television, and summarizes the 
interim conclusions from Salvador and the others on the effect of 
these decisions* The concluding pages are thus a summary of what 
seema to have lieen learned from El Salvador's experience in light 
of comparable experience elsewhere* 

The El Salvador case is thus an example of an uncommonly 
full case report, based on an uncommon amount of available hard 
research and administrative history. It was written by the head of 
the research group who had been studying the project for over three " 
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y6ar8« Inasmuch as the other members of the team participated in 
the work that vent into the study » and read it critically in 
manuscript I one might say that the study represents something 
over seven man^-years spent trying to understand and evaluate the 
project that was the focus of the case. Heedless to say, this is 
not the way most case studies are made. 

A one-man study where information was scarce . Let us, 
therefore, turn to actudy at the far end of the case study continuum. 
This is a report by Kimmel (1971) on Mew Zealand* e use of instructional 
television* 

The study was made by the author, . a member of an educational 
research and training institute in Washington, D. C., in a little 
less than two weeks, during which time he visited three principal 
cities of Mew Zealand. Little documentation and virtually no 
research were available, and the chief sources of information were 
officials of the Ministry of Education. The chief source in print 
was a paper by a New Zealand educator on the radio-^correspondence 
courses, published by UHESCO in 1967. Probably because this part 
of the program was covered in geater detail by the 1967 paper ^ Dr. 
Kimmel. spent more time on the school broadcasts. 

His ten^page paper begins with a brief account of the New 
Zealand educational system, followed by two pages on the history 
of educational radio in that country. He describes the schedule 
of broadcasts (about an hour a day), and then takes up the foggy 
question of how many schools and correspondence Students use the 
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programs. The one piece of evidence that appeared to be available 
was a questionnaire survey to which 95 schools responded, suggesting 
that the range of use for different programs was from 6 to 28 
per cent) with the median around 10 per cent* There seemed to be 
no hard evidence as to use of the programs by the correspondence 
students, who had no teacher with them to decide whether the 
educational broadcasts should be tuned In. Nor was there ahy real 
evidence of the effects of the programs, which Is studied, the 
author says, by "questionnaires, In-servlce teacher training exer** 
clses, school visits by the production staff > and surveys of members 
of teachers* organizations.** Costs? The annual budget of the 
educational division of the New Zealand Broadcasting Corporation 
Is $70,000; **although no figures are available,'* the author says 
rather ruefuUyi **the estimated cost of producing a single program 
Is $100.** 

It Is evident that this case study must have been a frus* 
tratlng experience, and the result was the kind of study that Is 
useful only as a first look for ascertaining what New Zealand 
Is doing with Instructional radloi without understanding it or Its 
effect very deeply^ 

> What could have been done, given the apparent absence of 
research and documentary evidence? Perhaps nothings and we are 
not meaning to Imply tha^ Dr. Klmmel should have done any more than 
he did. Yet the problem of how to handle a situation where the 
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evidence is so scant is a contmon problemi and we can veil spend 
a little time on It. tt la possible that one or two things 
might have been done to enrich the case of New Zealand beyond 
what finally came out. One might have added members to the team 
with specialized knowledge^ In order to look at more specialized 
questions. For examplei an economist might have been able to dig 
out some useful cost figures. Another possibility would have been 
to focus the case on some smaller and more Interesting topic than 
the whole of New Zealand's Instructional radio. If, for example > 
the visitor or his team had chosen to concentrate on the home 
radio-correspondence students, and add to the contribution of the 
1967 report, he might have been able to visit a sample of the homes 
where these students are working, or to collect some data on how 
many had graduated from the correspondence course and what success 
thay had In later education, or perhaps to study New Zealand's 
technique In combining radio with correspondence material- 
He are going to conclude this paper with examples of how 
these two things were done In other cases where the Information 
was little, If any, more promising than In Mew Zealand. 

A three-man case study where Information was scarce . In 
1966 a case study team, made up of an American media researcher, 
a Latin American educator , and a European economist who had lived 
some time in lAtln America, went to Honduras to look at the church- 
sponsored radio schools of that country, which are modeled after 




36 



the Radio Sutatonza program In Colombia* This team also stayed 
a little less than two weeks » and wrote a report only a little 
longer than the one on New Zealand. They too found that research 
was almost non-existent and very little Information was available 
on paper. But they got more data out of the experience, due, very 
possibly, to the varied makeup and the size of the team. 

The economist dug Into cost records, and developed at least \ 
a rudimentary estimate of what the radio school actually cost, in 
total, and per student« The educator and the researcher Investi- 
gated the administrative arrangements and assembled estimates of 
the audience served by the schools over a period of five years, 
the number of students who took the examinations and the proportion 
who passed. The team leader uncovered the only known piece of 
research on the effectiveness of the schools 108 Interviews with 
former students. The educator examined closely some of the final 
examinations given by the school, and concluded that the officially 
estimated proportion of passes may be doubtful. 

This is fragile evidence , and not much evidence, but It ' 
advances our knowledge of these radio schools substantially. For 
the first time in print, the study provided an estimate of cost, 
audience served, and at least some evidence that students viere learn* 
ing. In addition, the case report included a discussion of some of 
the factors making for success or failure of the program. Thus the 
result was more than a casual reporting job; It threw a little light 
g belpv ; the^^^^^j 
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A focused study . Just as a team study has certain advan- 
tages over a one«man 8tudy» a focused studyi when information is 
scarce) may have some advantage over a general study. 

A focused study was made in India in 1966 by an American 
researcher and two Indian associates. The researcher had gone to 
India to find out what had happened to the radio rural forum since 
the pilot trials of the forum » ten years earlier. Those trials had 
been a rather sensational successi and had been reported in a book 
by Mathur and Neurath, as we indicated earlier in this paper. But 
very little had been reported since* When the American got to India 
he discovered that a great deal had happened. After several years 
of delay India had decided to to expand the forums over the country^ 
and had set an initial goal of 2S>000 forums to be established in 
five years. This goal had never been reached -"• in fact* the 
number had barely reached half the goal. There were murmur in gs of 
dissatisfaction with the forums. 

Recognizing a problem of practical import when he saw lt» 
the researcher decided to focus the studyi not on the forums in 
general) but on the attempt of India to expand them nation*wide» 
What exactly had happened? If the expansion was no more successful 
than rumoredi what had gone wrong? And what could other developing 
countries leatn from the India experience? This was a more Interes? 
ting aad in some respects an easier task than the general kind of 
study done In New Zealand i because it permitted the researchers to 
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focus th6lr efforts rather than diffusing thero» and to organise 
their findings around a question that was certain to attract 
Interest* 

The American researcher associated two Indians vlth hln In 
the case study. These were a rural broadcaster and a rural exten- 
sion specialist, both of whom had been associated with the early 
history of the radio rural forum. Together they managed to put 
together an accounting of the number of forums that actually had 
been established In each state since the goal of 25,000 had been 
aet (the figure was 12,776, of which several thousand were apparently 
Inactive). From cost records they made an estimate of the total 
budget put Into the forums, the cost of establishing an active 
forum (a little over $100 If 500 were established, about $75 In groups 
of 2,000)) and even the cost of getting one action project accepted 
(about $37 If each forum adopted two such projects a year, about 
$10 If each forum adopted 10 projects). 

In addition, they found six unpublished evaluative reports 
on the forums. None of these was very rich In data, but each one 
contributed some evidence, and one of them contained very useful 
findings on the number and kind of action projects adopted by the 
forums In each of five states during 1963. Furthermore, because 
th^ studies were done In different parts of India, they could be 
used to some extent to check whether what was happening seemed to 
be a national or a local phenomenon. 
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On the basis of these datai plus visits to some of the 
presently active forums and discussions with thelv leaders and 
utembers » the case study team was able to put together a considerable 
amount of material that had never been In print before, and more 
Impottanti to write a section on '^Lessons to be learned from the 
Indian experience/' The most Important of these was that a country 
must work as hard to expand a pilot project as on the initial 
project itselfi 

In conclusion 

It Is clear that there are ways to Improve case studies* 
Yet we feel that the case Is a useful form In which to study 
instructional media. A case study Is not something one does Instead 
of a field experiment on a significant project or problem. The 
case form is especially usefuli however » in making known the facts 
about a project before it can be studied extensively by more 
rigorous methods^ And even if ^^e were able, sayi to double the 
number of experiments and quasl^-experiments we can do on instruc- 
tional media projects in the field, still case studies would be 
useful because of their realistic and whollstlc quality the 
fact that they can report on a project as a whole and put separate 
findings into context and proportion. 
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